Dynamic Active Constraints for Surgical Robots using Vector Field Inequalities

Robotic assistance allows surgeons to perform dexterous and tremor-free procedures, but robotic aid is still underrepresented in procedures with constrained workspaces, such as deep brain neurosurgery and endonasal surgery. In these procedures, surgeons have restricted vision to areas near the surgical tooltips, which increases the risk of unexpected collisions between the shafts of the instruments and their surroundings. In this work, our vector-field-inequalities method is extended to provide dynamic active-constraints to any number of robots and moving objects sharing the same workspace. The method is evaluated with experiments and simulations in which robot tools have to avoid collisions autonomously and in real-time, in a constrained endonasal surgical environment. Simulations show that with our method the combined trajectory error of two robotic systems is optimal. Experiments using a real robotic system show that the method can autonomously prevent collisions between the moving robots themselves and between the robots and the environment. Moreover, the framework is also successfully verified under teleoperation with tool-tissue interactions.Comment: Accepted on T-RO 2019, 19 Page

MBAPose: Mask and Bounding-Box Aware Pose Estimation of Surgical Instruments with Photorealistic Domain Randomization

Surgical robots are controlled using a priori models based on robots' geometric parameters, which are calibrated before the surgical procedure. One of the challenges in using robots in real surgical settings is that parameters change over time, consequently deteriorating control accuracy. In this context, our group has been investigating online calibration strategies without added sensors. In one step toward that goal, we have developed an algorithm to estimate the pose of the instruments' shafts in endoscopic images. In this study, we build upon that earlier work and propose a new framework to more precisely estimate the pose of a rigid surgical instrument. Our strategy is based on a novel pose estimation model called MBAPose and the use of synthetic training data. Our experiments demonstrated an improvement of 21 % for translation error and 26 % for orientation error on synthetic test data with respect to our previous work. Results with real test data provide a baseline for further research.Comment: 8 pages, submitted to IROS202

Autonomous Coordinated Control of the Light Guide for Positioning in Vitreoretinal Surgery

Vitreoretinal surgery is challenging even for expert surgeons owing to the delicate target tissues and the diminutive workspace in the retina. In addition to improved dexterity and accuracy, robot assistance allows for (partial) task automation. In this work, we propose a strategy to automate the motion of the light guide with respect to the surgical instrument. This automation allows the instrument's shadow to always be inside the microscopic view, which is an important cue for the accurate positioning of the instrument in the retina. We show simulations and experiments demonstrating that the proposed strategy is effective in a 700-point grid in the retina of a surgical phantom. Furthermore, we integrated the proposed strategy with image processing and succeeded in positioning the surgical instrument's tip in the retina, relying on only the robot's geometric information and microscopic images.Comment: Accepted on T-MRB 2022, 16 page

Single-Shot Pose Estimation of Surgical Robot Instruments' Shafts from Monocular Endoscopic Images

Surgical robots are used to perform minimally invasive surgery and alleviate much of the burden imposed on surgeons. Our group has developed a surgical robot to aid in the removal of tumors at the base of the skull via access through the nostrils. To avoid injuring the patients, a collision-avoidance algorithm that depends on having an accurate model for the poses of the instruments' shafts is used. Given that the model's parameters can change over time owing to interactions between instruments and other disturbances, the online estimation of the poses of the instrument's shaft is essential. In this work, we propose a new method to estimate the pose of the surgical instruments' shafts using a monocular endoscope. Our method is based on the use of an automatically annotated training dataset and an improved pose-estimation deep-learning architecture. In preliminary experiments, we show that our method can surpass state of the art vision-based marker-less pose estimation techniques (providing an error decrease of 55% in position estimation, 64% in pitch, and 69% in yaw) by using artificial images.Comment: Accepted on ICRA 2020, 7 page

Active Constraints using Vector Field Inequalities for Surgical Robots

Robotic assistance allows surgeons to perform dexterous and tremor-free procedures, but is still underrepresented in deep brain neurosurgery and endonasal surgery where the workspace is constrained. In these conditions, the vision of surgeons is restricted to areas near the surgical tool tips, which increases the risk of unexpected collisions between the shafts of the instruments and their surroundings, in particular in areas outside the surgical field-of-view. Active constraints can be used to prevent the tools from entering restricted zones and thus avoid collisions. In this paper, a vector field inequality is proposed that guarantees that tools do not enter restricted zones. Moreover, in contrast with early techniques, the proposed method limits the tool approach velocity in the direction of the forbidden zone boundary, guaranteeing a smooth behavior and that tangential velocities will not be disturbed. The proposed method is evaluated in simulations featuring two eight degrees-of-freedom manipulators that were custom-designed for deep neurosurgery. The results show that both manipulator-manipulator and manipulator-boundary collisions can be avoided using the vector field inequalities.Comment: Accepted on ICRA 2018, 8 page

Virtual Fixture Assistance for Suturing in Robot-Aided Pediatric Endoscopic Surgery

The limited workspace in pediatric endoscopic surgery makes surgical suturing one of the most difficult tasks. During suturing, surgeons have to prevent collisions between tools and also collisions with the surrounding tissues. Surgical robots have been shown to be effective in adult laparoscopy, but assistance for suturing in constrained workspaces has not been yet fully explored. In this letter, we propose guidance virtual fixtures to enhance the performance and the safety of suturing while generating the required task constraints using constrained optimization and Cartesian force feedback. We propose two guidance methods: looping virtual fixtures and a trajectory guidance cylinder, that are based on dynamic geometric elements. In simulations and experiments with a physical robot, we show that the proposed methods achieve a more precise and safer looping in robot-assisted pediatric endoscopy.Comment: Accepted on RA-L/ICRA 2020, 8 Pages. Fixed a few typo

A New Navigation System for Minimally Invasive Total Knee Arthroplasty

A computer-assisted navigation system to be used for total knee arthroplasties (TKAs) was reported to improve the accuracy of bone resection and result in precise implant placement, but the concomitant surgical invasion and time consumption are clinical problems. We developed a computed tomography (CT)-based navigation system (NNS) to be used for minimally invasive TKA. It requires only the reference points from a small limited area of the medial femoral condyle and proximal tibia through a skin incision to obtain optical images. Here we evaluated the usefulness and accuracy of the NNS in comparison with the commercially available BrainLAB image-free navigation system (BLS). In a clinical experiment, the registration times obtained with the NNS tended to be shorter than those obtained with the BLS, but not significantly so. The NNS group tended to be in the extended position in the sagittal plane of the distal femur within 3 degrees, and the BLS group showed rather flexed deviation in the sagittal plane of the anterior femur

An Analysis of the Characteristics and Improved Use of Newly Developed CT-based Navigation System in Total Hip Arthroplasty

We developed a surface matching-type computed tomography (CT)-based navigation system for total hip arthroplasty (the N-navi; TEIJIN NAKASHIMA MEDICAL, Okayama, Japan). In the registration step, surface matching was performed with digitizing points on the pelvic bone surface after coarse paired matching. In the present study, we made model bones from the CT data of patients whose acetabular shapes had various deformities. We measured the distances and angles after surface matching from the fiducial points and evaluated the ability to correct surface-matching registration on each pelvic form, using several areas and numbers of points. When the surface-matching points were taken on the superior area of the acetabulum, the correction was easy for the external direction, but it was difficult to correct for the anterior and proximal directions. The correction was difficult for external and proximal directions on the posterior area. Each area of surface-matching points has particular directions that are easily corrected and other directions that are difficult to correct. The shape of the pelvis also affected the correction ability. Our present findings suggest that checking the position after coarse paired matching and choosing the surface-matching area and points that are optimal to correct will improve the accuracy of total hip arthroplasty and reduce surgical times

Novel and Simple Ultrasonographic Methods for Estimating the Abdominal Visceral Fat Area

Objectives. To evaluate the abdominal visceral fat area (VFA), we developed novel ultrasonographic (US) methods for estimating. Methods. 100 male volunteers were recruited, and their VFA was calculated by two novel US methods, the triangle method and the ellipse method. The VFA calculated by these methods was compared with the VFA calculated by CT. Results. Both the VFA calculated by the triangle method (r=0.766, p<0.001) and the ellipse method (r=0.781, p<0.001) showed a high correlation coefficient with the VFA calculated by CT. Also, the VFA calculated by our novel methods were significantly increased in subjects with one or more metabolic risk factors than in those without any risk factors. Furthermore, the correlation coefficients obtained using the two methods were enhanced by the addition of multiple regression analysis (with the triangle method, r=0.8586, p<0.001; with the ellipse method, r=0.8642, p<0.001). Conclusions. The VFA calculated by the triangle or ellipse method showed a high correlation coefficient with the VFA calculated by CT. These US methods are easy to use, they involve no radiation exposure, and the measurements can be conducted frequently. We hope that our simple methods would be widely adopted for the evaluation of VFA